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Abstract: This study sought to investigate four separate issues regarding student performance in a blended learning 
environment in the delivery of a Psychology course to 140 University undergraduates. Firstly, to investigate the 
relationship between student performance on three different coursework assignments and their performance on interim 
online assessments. Secondly, to investigate the relationship between student performance on coursework assignments 
and their perceptions of online community. Thirdly, to investigate the relationship between students’ online community 
scores and their engagement with an online assessment. Finally, to look at the relationship between students’ cognitive 
information processing style and coursework performance. The findings revealed that student coursework performance 
was positively related to their engagement with the online assessments. Furthermore, student online community scores 
were related to their coursework performance as was cognitive style. The results are discussed in terms of a 


consideration of these three factors in course design within a 
Keywords: Community, cognitive style, interim assessment 

1. Introduction 

The initial rationale underpinning the use of an 
interim online assessment system is to monitor 
and identify, early on in a course, those students 
who appear to not understand the course 
material. The most expedient way to achieve this 
is through the use of online multiple-choice tests 
(MCQs). Students obtaining low scores here can 
then be offered additional help and instruction. 
Furthermore, such a system could also be useful 
as a way of identifying students who are failing to 
attend classes, or students who are attending 
classes less regularly than other students. 

One further possible reason for student success 
or failure on a particular course is the degree to 
which these students engage with that course and 
with the other students on the course. This sense 
of engagement between learners is referred to as 
sense of classroom community. Rovai (2002) 
offers an explanation of sense of classroom 
community as being mutual interdependence and 
a sense of trust and interaction among community 
members. This means that the members of the 
community have shared goals and values. Online 
community needs to be thought of in terms of the 
activities people perform together in their group 
and not physically where they perform such 
activities. There are four components of 
classroom community outlined by Rovai (2001) 
and these are described as follows. Firstly, ‘spirit’ 
which is the feeling of belonging to and 
acceptance of a group identity. This refers to the 
recognition of membership of a community and 
the feelings of cohesion that develop amongst 
learners in a group as a result of this. Secondly, 
‘trust’ is simply the feeling that the group can be 
trusted and the group members will give feedback 

ISSN 1479-4403 127 


blended learning framework. 


to each other. Once this is established, members 
of the group or community can speak with 
confidence to other members of the group. 
Thirdly, ‘interaction’ is the feeling that community 
members have that they may benefit by 
interacting with other members of the community. 
Finally, ‘learning’ is the sense that community 
members have that learning can come about due 
to the community discussing information, that is, 
the community can construct knowledge. In a 
typical online environment group members may 
engage in interactive behaviour such as 
discussion, exchange of ideas and seeking 
advice. 

1.1 Community 

There is a body of evidence that would argue in 
favour of the importance of a sense of community 
in learning. For example, Wang (2001) argues 
that community can also result from shared 
knowledge among learners in an online 
environment, and the evidence suggests that 
interaction between learners in online courses 
could be of great importance to learner success 
(e.g. Shale and Garrison, 1990). Indeed, Citera 
(1998), suggests that online discussions 
encourage more reticent individuals to participate 
to a greater extent, while Warschauer (1997) 
advocates interaction in online environments, 
where there is less opportunity for intimidation 
between individuals and, in asynchronous 
settings, less time pressure than in face-to-face 
interactions. However, the lack of a sense of close 
interaction between learners in an online 
environment may have adverse consequences, 
which may be because learners experience 
feelings of isolation. Haythornthwaite, Kazmer, 
Robbins and Shoemaker (2000), suggested that 
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the participants in their study who failed to make 
online connections with other learners in their 
group reported feeling isolated and more stressed 
than those who made more connections. It would 
seem therefore that a strong sense of community 
and interaction is also crucial in online learning 
environments. 

1.2 Individual differences 

Although the studies cited above indicate the 
extent to which sense of community is necessary 
for effective learning in both face-to-face and 
online environments, it is pertinent to recognise 
that individual difference factors have also been 
shown to have an effect on sense of classroom 
community. For example, Rovai (2002) has 
demonstrated that there are differences in sense 
of classroom community between males and 
females, with females reporting a greater sense of 
community, and Kim and Bonk (2002) have 
demonstrated differences in sense of community 
cross-culturally, with students from the USA 
displaying a greater sense of community than 
those from Finland. Recently, Graff (2003) has 
provided some evidence to suggest that sense of 
classroom community is related to an individual’s 
learning style, with those possessing a globally 
orientated, socially dependent learning style 
reporting a lower sense of community than those 
possessing an analytic less socially dependent 
style. Accordingly, it is also possible that the way 
in which a student learns, may be identified by an 
assessment of individual difference factors, and 
one of the most profitable areas in this case is the 
notion of cognitive style. 


1.3 Research questions 

Four specific research questions were tested 
here. Firstly, what is the relationship between 
performance on coursework assignments and 
performance on interim online assessment? 
Secondly, is there a relationship between 
performance on assignments and classroom 
community scores? Thirdly, what is the 
relationship between classroom community 
scores and MCQ engagement? Finally, what is 
the relationship between cognitive style and 
coursework performance? 

2. Method 

2.1 Participants 

Participants in this study were 140 first year 
undergraduate students. These were 25 males 
and 115 females ranging in age from 18 to 54 with 
a mean of 23.15 and an SD of 8.11. 

2.2 Assessment methods 

2.2.1 Online multiple choice questions 

After each lecture, students were required to 
attempt several online multiple-choice questions 
based on the content of the lecture. The questions 
were posted on Blackboard, an example of which 
is shown in Figure 1 below. Correct responses to 
the multiple-choice questions yielded a normal 
distribution. 
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Name: Lecture 2 quiz 

Instructions: The same as for the previous quiz, simply read the question and click on what you think is the most appropriate response 

Multiple Attempts: NOT ALLOWED. This test may only be taken once. 

Force Completion: This test may be saved and resumed later. 

Ouestion 1 Multiple Choice 1 points 

In interview sitations, interviewers tend to ask 

O questions on biographical information 
O questions on the interests of the applicant 
O different questions to different candidates 
O all of the above 


Ouestion 2 Multiple Choice 1 points 

In the study of Forsythe, Drake and Hogan (1985), it was found that 

O job candidates who wore traditional dress were rated more highly 

O job candidates who wore the second most masculine type of clothing were rated as more decisive and self-reliant 
O wearing feminine clothing was rated more highly by female interviewers 
O spectacles made candidates more attractive 

<£) Powered by Blackboard £ Internet 

Figure 1: Example multiple-choice questions 
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2 . 2.2 Coursework assignments 

Three different types of assessment were utilised 
in this course. Theses were: 

1. A short essay 800 words. 

2. A critical evaluation of a psychology article. 

For this, students had to choose an article 
and write about the psychological implications 
of the article’s content from their knowledge of 
psychology. This was not meant to be an 
essay, but merely a commentary. 

3. A long essay 1500 words. 

2.3 Psychometric instruments 

2.3.1 Cognitive styles index (Allinson and 
Hayes 1996) 

The Cognitive Styles Index (CSI) (Allinson and 
Hayes 1996) is a self-report test designed to 
measure the analyst-intuitive dimension of 
cognitive style. The term intuitive is used to 
describe an individual who makes judgements 
based on feelings and who adopts a global 
approach to processing information, whereas the 
term analytic describes an individual who makes 
judgements based on reason, and who focuses 
on specific detail when processing information. 
Analysts tend to be inward looking and self-reliant, 
whereas intuitives are more socially directed 
orientation and look more to others. The CSI 
contains 38 statements, to each of which a 
respondent must indicate a true/uncertain/false 
response. It has a theoretical maximum score of 
76. Higher scores indicate a more analytical 
cognitive style and lower scores indicate a more 
intuitive style. 

The psychometric properties of the CSI instrument 
are documented by Allinson and Hayes (1996). 
From a sample of 1000 participants, they report a 
mean score of 38.5. Furthermore, graphical 
inspection and a Kolmogorov-Smirnov ‘goodness 
of fit’ test suggest that the test scores are normally 
distributed. Test-retest reliability of the instrument 
is also sound (r = 0.90, p <0.001), and mean 
scores of 34.60 and 35.40 indicate no significant 
changes over time (t = 0.82, p > 0.05). Finally, 
internal consistency scores measured by 
Cronbach’s Alpha taken from seven independent 
samples range from .84 to .92. The construct 
validity of the CSI has also been reported by 
Allinson and Hayes (1996). They report 
statistically significant relationships between the 
CSI and scores on the Myers Briggs Type 
indicator. For example, the CSI correlated 
positively with the extroversion - introversion (r = 
0.57, p < 0.001), dimension and correlated 
negatively with the sensing - intuitive (r = - 0.41, p 
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< 0.05) and judgement - perception (r = - 0.41, p < 
0.01) dimensions of this instrument. 

2.3.2 Classroom community index (Rovai, 
2002 ) 

The Classroom Community Index is a self-report 
instrument consisting of 40 items, requiring a 
response on a 5-point Likert-scale ranging from 
strongly agree to strongly disagree. Scores on the 
questionnaire range from 0-160, with low scores 
reflecting a weak sense of community and high 
scores reflecting a strong one. The questionnaire 
also features four subscales of spirit, trust, 
interaction and learning with 10 items measuring 
each and scores ranging from 0 to 40. Rovai 
(2002) reports a high degree of face validity of the 
instrument, in that the items appear to measure 
what is needed to assess community. Internal 
consistency estimates using Cronbach’s alpha as 
reported by Rovai (2001) reached .96 for the total 
scale, .90 for the spirit subscale, .84 for the trust 
subscale, .84 for the interaction subscale and .88 
for the learning subscale. 

3. Results 

3.1 The relationship between 
performance on coursework 
assignments and performance on 
the MCQs 

Clearly students who had attempted more of the 
MCQs would have scored more than those who 
attempted less. Therefore for the purpose of data 
analysis, the mean and not total MCQ scores 
were used, and placed in the following categories. 
MCQ scores of between 1 and 3.4 were assigned 
to category 1, scores between 3.5 and 3.9 were 
assigned to category 2 and scores between 4 and 
5 were assigned to category 3. This system of 
data analysis ensured that each category 
contained approximately 30% of participants. A 
one-way analysis of variance was performed on 
each assignment score and the mean assignment 
score for each MCQ category. However, no 
significant effects were observed. 

3.2 The relationship between 
performance on coursework 
assignments and engagement with 
the MCQs 

The analysis above illustrates that performance 
on coursework assignments was not related to 
mean performance on the MCQs, and the next 
part of the analysis assesses whether 
engagement, that is the number of MCQ tests 
attempted by participants was related to 
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performance on their coursework assignments. 
The MCQs were set after each of a series of six 
lectures and therefore students could have 
attempted anything between 0 and 6 MCQs. A 
one-way analysis of variance was performed on 
each assignment score for the number of MCQs 
attempted. A significant effect was evident only for 
coursework assignment three (F, (82, 5) = 2.49, p 
= 0.04). Figure 2.shows coursework assignment 
scores for each MCQ category for assignment 3 
and illustrates that those who attempted 4 or more 
MCQs scored higher on coursework assignment 
3, whereas those who attempted 3 or less scored 
lower on coursework assignment 3. 



Figure 2: Scores on coursework assignment 3 
and MCQs attempted 


3.2.1 The relationship between performance 
on coursework assignments and 
classroom community scores 

In order to investigate the relationship between 
coursework assignments and classroom 
community scores, the community scores were 
subdivided into three categories, low community 
scores (category 1), medium community scores 
(category 2) and high community scores (category 
3). A one-way analysis of variance was performed 
for coursework assignment scores for each 
category of the classroom community 
questionnaire, (that is high, medium and low). An 
effect approaching significance was observed for 
the spirit subcategory of the community scale and 
only for assignment 3, (F (2, 72) = 2.63, p = 0.07). 
This is illustrated graphically in Figure 3. No 
significant effects were observed for coursework 
assignment scores for any of the other community 
categories. 



Figure 3: Scores on coursework assignment 3 
and Spirit Category 

3.3 The relationship between community 
scores and MCQ engagement 

The third research question looked at the 
relationship between community scores and MCQ 
engagement. A one-way analysis of variance was 
performed on the community scores for each 
category of MCQ engagement as outlined above. 
However, no significant effects were observed for 
MCQ engagement and community scores. 

3.4 The relationship between cognitive 
style and performance 

The final research question centred on an 
analysis of the relationship between cognitive 
style and coursework performance. For the 
purpose of data analysis in this study, cognitive 
style was divided into three categories. A one-way 
analysis of variance was carried out, and revealed 
a significant effect between coursework 
performance and cognitive style, only for 
coursework assignment 3 (F (2, 36) = 3.67, p = 
0.03). In this case, analytics scored highest. This 
is shown in more specifically in Figure 4. 



Figure 4: Scores on coursework assignment 3 
and cognitive style category 
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4. Discussion 

This study sought to investigate several issues 
related to interim assessment and the results are 
discussed in the following. Firstly the results 
showed no relationship between performance on 
the coursework assignments and performance on 
the online MCQs. The MCQ categories were very 
narrow and this may account for the lack of any 
observable differences here. What is possibly 
more critical is the relationship between 
performance on the coursework assignments and 
engagement with the MCQ system. Here the 
results showed a difference in that students who 
attempted 4 or more of the six MCQ tests scored 
better on assignment 3 than those who attempted 
3 or less MCQs. This would appear to suggest 
that students who perhaps engaged more with the 
course generally were likely to perform better in 
coursework. Why this is only true for coursework 3 
is possibly because this coursework was a more 
substantial piece of work than the first two. 

This study also sought to look at the concept of 
classroom community, and the relationship 
between this and performance on coursework 
assignments. The results showed that students 
with a high spirit score on the community index 
performed better on coursework assignment 3. As 
described above, ‘spirit’ is the feeling of belonging 
to and acceptance of a group identity. This refers 
to the recognition of membership of a community 
and the feelings of cohesion that develops 
amongst learners in a group as a result of this. It 
appears then that a feeling of belonging to a 


group is essential for good performance. 
However, there would appear to be no 
relationship between classroom community 
scores and MCQ engagement. In line with 
studies, which show a relationship between 
cognitive style and academic performance, this 
study also illustrated such a relationship. Here 
analytics as defined by the Cognitive Styles Index 
performed best on assignment 3, with no 
difference in performance between learners of 
different cognitive styles in the other coursework 
elements. The consistent finding here is for a 
difference to be evident on assignment 3. As 
mentioned above, assignment 3 was a more 
substantial piece of work, and this may account 
for the difference observed here. A further 
possible interpretation is that differences in ability 
between students were beginning to become 
evident at the time they took the third assignment. 
However, it also possible that assignment 3 was 
an assignment, which differentiated between 
students on ability, and this, is something, which 
is worthy of further investigation. 

5. Conclusion 

In conclusion, the general finding from this study 
showed several factors determined good 
performance on coursework assignments. These 
were engagement with the course in terms of the 
amount of online assessment attempted, 
classroom community, and cognitive style. It 
would therefore seem reasonable to consider 
these factors in the design of course material. 


References 

Allinson C. W. and Hayes, J. (1996) The cognitive styles index: A measure of intuition-analysis for organisational 
research, Journal of Management Studies, 33(1), 119-135. 

Citera, M. (1988) Distributed teamwork: The impact of communication media on influence and decision quality, Journal of 
the American Society for Information Science, 49, (9), 792-800. 

Graff, M. G. (2003) Individual differences in sense of classroom community in a blended learning environment, Journal of 
Educational Media, 28 (2-3) 203-210. 

Haythornwaite, C., Kazmer, M. M., Robbins, J and Shoemaker, S (2000) Community Development among distance 
learners: Temporal and Technological Dimensions, Journal of Computer Mediated Communication, 6, (1), 1-24. 

Kim and Bonk (2002) Cross-cultural comparisons of online collaboration, Journal of Computer Mediated Communication, 
8, (1) available on-line http://www.ascusc.org/jcmc/vol8/issue1/kimandbonk.html (accessed 1 May 2006). 

Rovai, A.P. (2001) Building classroom community at a distance: a case study. Educational Technology Research and 
Development Journal, 49 (4), 33-48. 

Rovai, A.P. (2002) Sense of community, perceived cognitive learning, and persistence in asynchronous learning 
networks. Internet and Higher Education, 5, 319-332. 

Shale, D. and Garisson, D. R. (1990) Education and Communication. In D. Garrison and D. Shale (Eds) Education at a 
distance: from issues to practice, 23-39. Malabar, FL: Robert E. Krieger. 

Wang, M. (2001) The construction of shared knowledge: in an Internet-based shared environment for expeditions 
(iexpeditions): A study of external factors implying knowledge construction Dissertation Abstracts International 
Section A: Humanities and Social Sciences, 62 (5-A). 

Warschauer, M. (1997) Computer-mediated collaborative learning: Theory and practice, Modern Language Journal, 8, 

(4), 470-481. 


www.ejel.org 


131 


ISSN 1479-4403 



Electronic Journal of e-Learning Volume 4 Issue 2 2006 (127 - 132) 


www.ejel.org 


132 


©Academic Conferences Ltd 



